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ABSTRACT 


Considering  synthetic  patterns  as  the  most  elementary  units 
of  semantic  information^  pattern  encoding  and  decoding  mechanisms  are 
discussed  as  the  basic  components  of  semantic  communication  systems. 
Employing  mathematical  concepts  long  used  "by  social  scientists  for 
modeling  spatial  interaction  patterns  within  complex  social  systems,  a 
particular  representation  of  pattern  information  is  presented  and,  with 
respect  to  this  notation,  a  numerical  measure  of  pattern  correlation  is 
developed  that  permits  transducer  recognition  of  internally  stored 
patterns  maximally  correlating  with  patterns  received  from  an  external, 
noisy  information  source.   Possible  generalizations  of  the  model  lead  to 
brief  discussion  of  the  potential  utility  of  the  concepts  employed  within 
the  development  of  macro  theories  of  natural  communication  processes. 


Introduction 

Attempting  to  formalize  in  both  philosophical  and  mathematical 
terms  the  primacy  of  information  flow  within  systems  exhibiting  purposive 
behavior,  Wiener  termed  cybernetics  the  study  of  all  communication  and 
control  processes,  whether  mechanical,  biologica],  or  social  in  nature.   '  '^ 
Not  independently  of  Wiener,  Shannon  established  an  outline  adequate  for 
conceptual  analysis  of  any  communication  system  and,  distinguishing  sharply 
between  message  semantics  and  message  encodings,  formulated  a  mathematical 

theory  of  transmission  efficiency  for  communication  channels  linking  mes- 

(k) 
sage  encoding  and  decoding  operations.     Inspired  by  the  utility  and 

elegance  of  Shannon's  concepts  and  mathematical  formulations,  Weaver  (and 

others)  immediately  speculated  that  the  contributions  of  Wiener  and 

Shannon  would  generalize  toward  a  more  comprehensive  theory  of  communi- 

(5) 
cations  treating  message  semantics  as  well  as  transmission  efficiencies. 

Despite  considerable  development  of  Wiener-Shannon  information  theory  to 

(67) 
date,^  ^  '^  as  well  as  parallel  developments  in  psychology,  linguistics, 

and  language  philosophy,    '      '  there  remains  lacking  any  viable  prin- 

ciple for  integration  of  existing  theory  toward  a  conceptual  framework 
of  sufficient  coherence  to  support  hypotheses  concerning  semantic  com- 
munications within  natural  systems.   '. 

Employing  Shannon's  premise  that  "the  fundamental  problem  of 
communication  is  that  of  reproducing  at  one  point  either  exactly  or 
approximately  a  message  selected  at  another  point,"  below  we  revise 
Shannon's  diagram  of  a  general  communication  system  to  emphasize  the  nature 
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of  the  encoding  and  decoding  operations  that  occur  at  either  end  of  a 
conmunication  channel.   Messages  conveyed  from  source  to  destination, 

however  complex,  are  necessarily  organized  in  terms  of  a  finite  vocab- 
ulary of  semantic  elements  common  to  "both  encoding  and  decoding  oper- 
ations alike.   Except  for  labels  and  schematic  indications  of  information 
flow,  the  symmetry  of  the  diagram  again  reflects  the  bi-directional  nature 
of  communications--processes  occurring  always  in  the  presence  of  noise. 


Information       / 
Source 


Information 
Destination 


G 


Encoder 


Decoder 


Figure  1.   Shannon's  schematic  diagram  of  a  general  communication  system. 


If  we  use  the  terra  pattern  generally  to  encompass  all  basic 
organizational  units  of  semantic  information  (signs,  symbols,  characters, 
words,  etc.),  we  may  regard  a  fundamental  goal  of  pattern  Information 
processing  research  to  be  the  specification  of' formal  models  of  the 
message  encoding  and  decoding  operations  that  enable  semantic  information 


transmission  between  message  source  and  destination.   A  primary 
theoretical  problem  of  message  encoding  requires  a  formal  representation 
of  quantized  semantic  patterns  appropriate  for  message  organization  and 
transmission.   A  corresponding  decoding  problem  requires  specification 
of  models  of  pattern  information  recognition  of  sufficient  efficacy  and 
generality  to  support  at  least  theoretical  estimation  of  the  degree  of 
system  complexity  necessary  for  message  comprehension.   At  a  minimum 
level  of  complexity,  the  problem  requires  definition  of  measures  of 
pattern  information  correlation,  similarity,  or  association  that  may 
be  employed  to  effect  accurate  transducer  addressing  of  those  patterns 
internal  to  Its  own  vocabulary  that  best  communicate  the  semantics  of 
the  information  transmitted  to  it  by  an  external  message  sovirce. 

The  diversity  of  techniques  currently  employed  as  pattern 
recognition  models  makes  difficult  any  simple  typology  of  methodologies. 
(See  references  (ik)   and  (15).)    . 

For  the  purposes  of  this  paper,  It  will  be  sufficient  to 
distinguish  only  two  broad  approaches  to  pattern  recognition  which  we 
will  refer  to  as  analytic  and  synthetic.   By  analytic,  we  mean  any 
methodology  for  which  pattern  classification  depends  either  upon  analysis 
of  component  pattern  parts,  attributes,  or  features  (e.g.,  statistical 
methods),  or  upon  analysis  of  structural  relationships  between  pattern 
components  (e.g.,  syntactic  methods).   By  synthetic,  we  mean  simply  any 
methodology  that  treats  patterns  as  Integral  units  of  information  and 
that  categorizes  patterns  In  accordance  with  similarities  determined  by 
methods  of  direct  comparison  (e.g.,  template  matching  techniques  and 
Wldrow's  "rubber-masking"  approach.^  '         ) 


The  inadequacies  of  traditional  correlation  techniques  for 
quantification  of  pattern  similarities  have  discouraged  greatly  to  date 
the  research  and  development  of  pattern  recognition  models  "based  on 
methods  of  direct  comparison.    >    ^^      '      On   the  other  hand,  the  rationale 
underlying  the  various  analytic  methodologies  so  widely  adopted  seems 
contrary  to  the  basic  notion  of  pattern  gestalt-- that  the  whole  is  more 
than  the  sum  of  its  parts.   The  perplexities  of  feature  extraction  and 
selection  processes  within  so  many  analytic  models  of  recognition  may  well 
attest  to  the  synthetic  nature  of  pattern  information  in  general.   One 
purpose  of  this  paper  is  to  suggest  that  pattern  information  processing 
research  may  have  abandoned  too  soon  the  search  for  quantitative  measures 
of  synthetic  pattern  similarity. 

Relying  greatly  on  mathematical  concepts  long  employed  by 
social  scientists  for  modeling  certain  aspects  of  organization  exhibited 
by  systems  of  social  interaction  within  community  spatial  structure,  be- 
low we  suggest  a  particular  representation  of  patterns  as  spatial  dis- 
tributions of  information  and  develop  a  mathematical  index  of  pattern 
information  correlation  expressible  for  any  two  patterns  so  defined.   The 
measure  of  pattern  correlation  is  first  developed  geometrically  for  pictorial 
patterns  as  a  coefficient  of  configuration  similarity  that  may  be  estimated 
invariant  with  respect  to  individual  pattern  intensities,  sizes,  positions, 
and  proximate  orientations.   This  particular  mathematical  formulation  of 
pattern  correlation  then  suggests  a  general  network  model  of  pattern 
information  processing  that  allows  representation  of  pattern  associa- 
tions as  network  distributions  of  information.   These  generalizations 


lead  to  consideration  of  the  particular  conabination  of  concepts  employed 
as  a  mathematical  basis  for  macro  theories  of  information  processing 
within  natural  communication  systems. 

Patterns  and  Pattern  Dissimilarities 

While  the  generality  surrounding  the  concept  of  a  pattern  makes 
difficult  any  single  verbal  definition,  to  assist  the  present  mathematical 
discussion  we  offer  the  following:   a  pattern  is  a  collection  of  quantized 
information  distributed  over  a  set  of  spatial  elements  characterizing  some 
more  complex  information  source.   If  we  adopt  at  least  provisionally  the 
above  definition,  we  may  represent  mathematically  a  particular  pattern  f  as 
a  partitioned  array  (w|x)  ,  as  tall  as  there  are  elements  of  f,  where  W 
is  a  matrix  of  coordinates  indicating  the  relative  spatial  positions  of  all 
elements  of  f  and  X  is  a  vector  of  positive  reals  indicating  the  propor- 
tional distribution  of  quantized  units  of  information — pattern  quits 
(quantization  units) — across  all  pattern  elements. 

For  mathematical  convenience  we  will  assume  normalization  of 

pattern  intensity  so  that  T  x.  =1.  Thus  X  may  be  considered  a  discrete 

i  1 

probability  distribution  of  pattern  quits  over  pattern  elements.   Also,  we 
will  assume  normalization  of  patterns  with  respect  to  positions  and  sizes  so 
that  pattern  centroids  are  coincident  at  a  common  origin,  )  .x,  w  ,,  =  0  for 
each  spatial  dimension  k,  and  second  moments  about  the  origin  equal  unity, 

With  such  mathematical  notation  we  consider  the  following  geometric 
pattern  recognition  problem:   given  a  set  of  normalized  patterns  F,  determine 
a  symmetric  non-negative  scalar  index  of  pattern  dissimilarity, 
^^  ^  =   6[(w|x)  ,  (y|z)  ],  pairwise  computable  for  all  feF  and  geF,  such  that 

-■-SB  I  g 


A    approaches  zero  as  the  similarity  of  f  and  g  increases.   Below  we 
develop  such  a  criterion  of  pattern  dissimilarity  that  is  by  normalization 
invariant  with  respect  to  individual  pattern  intensities,  sizes,  and 
positions  and  by  computation  optimal  with  respect  to  small  displacements 
of  individual  pattern  orientations. 

For  any  two  normalized  patterns  f  and  g  we  establish  an  index  of 
pattern  dissimilarity  in  the  following  manner.   We  determine  simultaneously 
a  weighted  correspondence  of  elements  between  f  and  g  and  a  registration  of  f 
with  respect  to  g  such  that  there  exists  maximal  spatial  congruence  of 
corresponding  elements  between  f  and  g.   We  then  take  as  our  criterion  of 
pattern  dissimilarity  the  weighted  sum,  of  squared  distances  between  all 
pairs  of  elements  corresponding  between  f  and  g  where  the  weights  of  the 
sum  reflect  the  degree  of  correspondence  between  each  element  pair. 

Let  the  two  patterns  f  and  g  consist  of  m  and  n  elements  respectively 
and  let  their  normalized  representations  be  denoted  (w| X) „  and  (y|z)  .   We 
represent  a  particular  weighted  correspondence  of  elements  between  f  and  g 
as  a  matrix  Q  (m^n)  satisfying 
m 


(i; 

L  q.     .    =   z. 
i      i;0           J 

a  =  ^, 

...,    n 

n 

(2) 

^^1,0^=^1 

i  =  1, 

.  .  .,   m 

(3) 

q.     .    >    0 

i  =  1, 

..  .,   m 

J  =  1,  .  .  . ,  n  . 

LetiTT    denote  the  set  of  all  Q,  matrices  satisfying  (l),  (2),  and  (3^  for 

given  X  of  f  and  Z  of  g.   Now  by  normalization  of  pattern  intensities 

E.x.  -  Y,.z .    =   1,  hence  E. >j.q.  .  -   1.   Since  also  q.  .  >  0  for  all  i  and  j, 


we  may  consider  any  Q  e  7t„   a  discrete  bivariate  probability  density  function 
characterizing  a  joint  distribution  of  corresponding  quits  over  the  elements 
of  f  and  the  elements  of  g. 

Let  a  particular  spatial  registration  of  f  with  respect  to  g 

be  denoted  oWR   +  JT'  where  J  is  the  vector  (l,  ...»  1  )',  T  is  a 

m 

translation  vector,  a  is  a  scale  factoij  and  R  is  a  rotational  trans- 
formation.  For  a  given  registration  of  f  with  respect  to  g,  let  S 
(m  X  n)  be  the  resulting  matrix  of  squared  Euclidean  distances  between 
the  elements  of  f  and  the  elements  of  g. 

j  =  1,.  ..  ,n 

Let  I  denote  the  set  of  S  matrices  obtainable  for  f  and  g  by  (k)   over 

all  positive  scalars  a,  all  translations  T,  and  all  legitimate  rotations 

R. 

Our  criterion  of  pattern  dissimilarity  may  then  be  formulated: 

m  n 
(5)   A    =    min  I   I   ^'    .  s.    =      min      tr(Q'S). 


le-n  SeJ      i  j   ^"^   ^"^   Qstt     SeJ 


Note  that  such  an  index  of  pattern  dissimilarity  may  be  interpreted  as  a 
minimal  mean  squared  error  of  registration  between  all  corresponding  pattern 
quits  of  f  and  g.  , 

Now  for  Q  given,  consider  all  possible  translations  of  f  with 
respect  to  g  and  write 

m  n 


(^'  '^i|%.,p"i,.-V--j,.i' 


To  determine  the  particular  T  that  minimizes  A  over  all  translations  of  f 
with  respect  to  g,  differentiate  (6)  with  respect  to  t,  to  obtain 

m  n 

from  which  it  may  be  determined  that  t^  =  0  where  /.  /  .  q.  .  w.  ,  = 
l-l-    %     .   y.  ,  ,  or  where  L    x.    x-r.  ^^  =  L  z  y    .   This  Implies  that  an 
optimal  registration  between  f  and  g  requires  a  coincidence  of  pattern 
centroids.   Since  by  normalization  pattern  centroids  already  occur  at  a 
common  origin  and  since  these  centroids  remain  invariant  over  pattern 
rotations  and  scale  .changes,  we  may  conclude  that  no  further  consideration 
of  pattern  translations  is  necessary  in  minimizing  A 

Now  consider  all  possible  positive  scale  factors  a  applied  to 
f  so  that 

m  n 

Differentiating  A  with  respect  to  a  we  find  that  the  particular  a 

minimizing  (8)  is  given  by 

m  n  m  n         p 

(9)  o  =    ill   0      -Ivy   ]/[l  I   q    ^.J  ]. 

If  o  scales  optimally  W  with  respect  to  Y^  then  by  symmetry,  a   scales 

optimally  Y  with  respect  to  W.   By  an  identical  analysis  we  may  determine 

for  a   the  expression 

-,    m  n  m  n 

(10)   a-l=  [lUi  ,  IvU^^^l   l-if,    I^lk^l  rI- 

1   J  1   J  ^ 


Henc 


eo  =  a-     =1  where  IJ^^   q^^^  l^   v.^j^  -  IJ^  q^^j  l^  t^^^,     or  where 

YyTw    =yz.  /nV^i-  Again,  by  normalization  this  condition  is 
^i  ^  ^k  i,k   ^j  j  ^k  "^J.k    ^ 

met,  since  pattern  second  moments  about  the  origin  are  set  to  unity.   Hence, 

in  minimizing  A    we  may  also  exclude  all  further  consideration  of  pattern 

scales. 

The  foregoing  analysis  demonstrates  that  the  pattern  positions 

and  sizes  yielded  by  normalization  are  optimal  with  respect  to  the  minimal 

A    independent  of  whatever  correspondences  Q  are  defined  between  the 
f  >6 

elements  of  f  and  g  and  independent  of  whatever  rotation  R  may  be  chosen 
to  effect  maximum  spatial  congruence  of  corresponding  elements  of  f  and  g. 
It  remains  to  determine  simultaneously  an  optimal  correspondence  matrix  Q 
and  a  rotation  R  of  f  yielding  an  optimal  registration  of  f  and  g  such 
that  the  minimal  A    may  be  determined. 

In  the  following  sections  we  present  a  computational  solution 
to  (5)  via  Iterative  solution  two  interdependent  subproblems — the 
correspondences  problem,  which  requires  minimization  of  A    over  all 

■*"  •  ft 

Qeir    for  fixed  S,  and  the  transformation  problem,  which  requires 

f  >8 

determination  of  a  rotation  R  that  minimizes  A„   over  all  SeT    for 

f»g  ijg 

fixed  Q. 

The  Correspondences  Problem 

Consider  f  and  g  as  two  patterns  taken  from  a  set  of  quantized 
patterns  that  may  be  assumed  to  be  normalized  with  respect  to  Intensity, 
position,  and  size  and  standardized  with  respect  to  orientation,  e.g.,  a 
font  recognition  problem.   Since  In  such  a  case  we  may  assume  no 
rotational  freedom  for  either  pattern,  we  may  consider  I  to  consist  of 

a  single  S  matrix  determined  directly  by 
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j   =  1,    . . . ,   n. 

In  such  an   instance,   our   formulation  of  pattern   dissimilarity  reduces  to 

m  n 

(12)      A  =  rain        T    y    q.     .    s.     .    =     min        tr    (Q'S) 

f,g  I   ^.      i,J      i,J 

where  again  ti^   is  the  set  of  all  Q  matrices  satisfying  (l),  (2),  and  (3). 
1  »g 

The  optimization  prohlem  given  by  (l),  (2),  (3),  and  (l2)  ,  where 

y  X  =  y   z  =  Y  but  Y  J^o't  necessarily  unity,  may  be  recognized  as  the 
^i   1   ^J   J 

(21  22  2^) 
Hitchcock  or  transportation  problem  of  linear  programming.    '  ■>    ~>J 

Typically,  the  problem  requires  determination  of  a  matching 

between  a  spatially  distributed  set  of  economic  supplies  and  a  spatially 

distributed  set  of  demands  such  that  the  total  cost  of  all  material 

movements  from  suppliers  to  buyers  is  minimal.   For  such  problems, 

computational  algorithms  are  well  known  and  solution  properties  well 

documented.   While  any  of  these  algorithms  may  be  used  for  computation  . 

of  A    in  the  present  case,  the  characteristics  and  computational 
f  ,g 

requirements  of  pattern  information  processing  in  general  compel  us  to 
look  further  for  mathematical  techniques  more  appropriate  for  parallel 
processing  systems. 

A  related  problem  known  to  transportation  planners  is  called 

(2^  25) 
the  entropy  network  distribution  problem.    '     The  problem  arises 

where  it  is  desirable  to  simulate  traffic  flows  within  a  metropolitan 
region  given  data  describing  distributions  of  populations  and  economic 
activities  over  some  set  of  analysis  zones  subdividing  the  region,  zone- 
to-zone  travel  times,  and  estimates  of  mean  travel  times  for  specific 
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types  of  trips  within  the  region.   Borrowing  the  notation  of  our  pattern 

correspondences  problem,  let  A  represent  the  mean  travel  time  for  all 

home-work  commuting  trips,  let  X  be  the  probability  distribution  of 

workers  over  m  residential  zones,  let  Z  be  the  distribution  of  jobs  over 

n  employment  zones,  and  let  S  be  a  matrix  of  network  travel  times  between 

any  residential  zone  and  any  employment  zone.   The  problem  requires 

determination  of  a  most  probable,  mean,  or  maximum  entropy  joint  probability 

distribution  Q  with  marginals  X  and  Z  such  that  each  element  q.    repre- 

sents  the  forecasted  proportion  of  all  trips  occurring  between  the  i-th 

residential  zone  and  the  j-th  employment  zone.   Mathematically,  the 

problem  is  formulated 

m  n 
(13)   max  H  =  -  J  J^  q    log  q 

i  j   ^"^      ^"J 

subject  to  (l),  (2),   and  (3)  and  the  additional  mean  travel  time  constraint 

m  n 

Note  that  constraint  (l^)  may  be  considered  simply  an  a  priori  specification 
of  overall  network  distribution  efficiency  or  energy  expenditure. 

The  solution  to  the  problem  is  given  by 

^15^   "li.j  =  \  \   ^j  ^j  ^^P  ^-  ^  ^i.,j^    ^  =  ^'  •••'  ^ 

j  =  1 ,  .  . .  ,  n 

where  3  represents  the  Lagrange  multiplier  associated  with  constraint  (ik) 

and  the  u  and  v  are  functions  of  the  Lagrange  multipliers  associated 

with  constraint  sets  (l)  and  (2).   It  has  been  shown      that  corresponding 
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to  any  real  3  there  exists  a  unique  Q  maximizing  (13 )  and  satisfying 

(l),  (2),    and  (3)  given  by  (15)  where  the  parameters  u.  and  v.  may  be 

J 

determined  by  iterative  solution  of  the  equations 


(l6)   u.  =[y  z.v.  exp  (-3s.  .)]      i=l,  ...,m 

^  1 

(IT)   V.  =  il  X.  u  exp  (-  3  s.  .)]      j  =  1,  ...,  n. 

.1  X    -L                 1  %  fl 


i 
Additionally,  it  has  been  shown  that  there  exists  a  monotonia  mapping 

between  all  3  and  all  feasible  A  such  that  as  3  approaches  -oo^  A  approaches 

A    ,  and  as  3  approaches  +°°,  A  approaches  A  .  ,  where  A    and  A  . 
max  mm         max      mm 

respectively  denote  the  maximum  and  minimum  values  of  A  possible  for  given 

(27  28) 
S,  X,  and  Z.     *     Together  these  results  yield  theoretical  justification  for 

iterative  determination  of  the  unique  Q  maximizing  (l3)  and  satisfying  the 

network  distribution  efficiency  constraint  (lU)  as  well  as  constraints 

(l),  (2),  and  (3).   Since  A  .   of  the  entropy  network  distribution  problem 

mm  ^''  -^ 

is  analogous  to  the  pattern  dissimilarity  criterion  of  o\ar  present  pattern 
recognition  problem,  these  results  also  imply  that  we  may  at  least  theoreti- 
cally formulate  a  maximum  entropy  correspondence  matrix  Q*,  unique  and 
optimal  with  respect  to  the  minimal  A    ,  via  equations  (15),  (l6!^  and  (iT) 

with  the  parameter  3  set  to  °°. 

(29  ) 
Using  theorems  developed  elsewhere     and  well  known  properties 

(28  )  .     ^  . 

of  the  Hitchcock  model   Evans      demonstrates  several  features  of  the  matrix 

Q*  and  suggests  a  strategy  by  which  it  may  be  computed.   Let  E  denote  a 
binary  matrix  such  that  e.  .  =  1  for  all  subscript  pairs  (i,j)  where 
q*  >  0  and  e.    =0  elsewhere.   (Properties  of- the  Hitchcock  model  imply 
that  E  will  be  sparse).   The  desired  Q*  and  the  matrix  E  then  interrelate 
arithmetically  in  the  form 
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(18)  q*_j  -  x^  u*  z^   V*  e^_j 

where  the  u.  and  v^  satisfy  the  relations 

(19)  u*  =  i[.j  ^j  n.jr^     i  =  i.  •••.■» 

*    "^     *      1 
(20)  V  =  [I   X  u  e   ]■     J  =  1,  ...,  n. 

J      -;-'--'■   1 » J 

Evans'  method  for  exposing  E,  and  hence  Q*,  requires  initially  a  solution 
to  the  Hitchcock  problem  (l2)  determined  presumably  by  traditional  methods. 
If  for  our  particular  entropy  network  problem  ve  may  assume  that 
convergence  of  •(l5)j  (l6),  and  (IT)  is  well-conditioned  for  large  3,  then 
a  matrix  Q*  may  be  computed  heuristically  in  the  following  manner.   Establish 
the  matrix  E  such  that  e.    =  exp(-  3s.   )  where  S  is  S  scaled  linearly 
to  have  elements  within  a  specified  interval  and  3  is  chosen  as  large  as 
computational  considerations  permit.   Determine  estimates  of  U*  and  V* 
via  iterative  solution  of 


n  ^      ^ 


(21)  u     =    [I   z     V     e        r  i   =  1.    ... 

^  ffl  ^      ^  _-, 

(22)  -j   =   qx,   u.    e.^.r  J=l,    ... 


,  m 


1 
and  then  estimate  Q*  via 


(23)      q.     .   =  X.    u.    z     V     e  i   =  1,    ...,  m 

i»J  1      1      j      j      i,j 

j   =  1,    . . .  ,   n. 


Now  assuming  that  3  is  sufficiently  large  such  that  Q  is  close 
to  Q*,  then  we  may  expect  small  elements  of  Q  to  correspond  to  zero 
elements  of  Q*.   Hence  we  may  select  for  each  q.    some  threshhold  value. 


li^ 


say  q.  .  =  x.  *  z  .  and  approach  E  by  resetting  e.  _,  =  0  whenever 

q.  .  <  q  for  any  particular  solution  to  (2l),  (22),  and  (23)j  that  is, 

1  5  J 


exp(-  6s.  .)ifq.  ._^q.  .   [   i  =  l. 


,  m 


e .  . 
""'"^     '  0  if  q     <  q     I   j  =  1,  ...,  n. 

Note  that  resetting  e.  .  =  0  at  any  point  is  equivalent  to  the  assumption 
that  pattern  elements  i  and  j  have  been  determined  non-corresponding  and 

thus  q    =0.   Since  initially  e.  .  is  taken  always  as  exp(-  6s.  .), 

i,j  ^'^  ^       ^"^ 

this  assumption  is  also  equivalent  to  the  condition  that  s.  .is  large 

relative  to  other  elements  of  S,  hence  e.  .  approaches  zero  faster  than 

other  elements  of  E  as  6  approaches  °°. 

Iterative  solution  of  (2l),  (22),  (23),  and  (2U)  in  the  above 
manner  should  lead  to  stable  values  for  Q  and  E.   At  convergence,  we  may 
equate  heuristically  Q  and  Q*.   Setting  all  remaining  positive  elements 
of  E  to  1  we  have  a  similar  approximation  of  E.   Q*  and  E  should  be 
related  via  (l8),  (l9),  and  (20). 

We  have  gained  some  evidence  that  such  a  procedure  is  sufficient 
for  determination  of  optimal  pattern  element  correspondences,  at  least 
where  elements  of  S  are  taken  as  squared  Euclidean  planar  distances  and  m 
and  n  are  small.   While  there  is  the  possibility  that  the  above  procedure 
will  prove  ineffective  (or  inefficient)  in  other  cases,  the  properties  of 
the  Hitchcock  and  entropy  network  models  offer  much  promise  for  the 
development  of  parallel  algorithms  practical  for  specific  applications. 
While  we  have  indicated  a  possible  course  for  the  development  of  such 
algorithms,  the  primary  intent  of  this  section  has  been  simply  to 
introduce  the  conceptual  utility  of  the  entropy  network  model  to  pattern 
information  processing  research. 
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In  this  context.  It  ahoiild  &lio  b*. noted  that  minljnu»-dl stance 

classification  of  patterns  using  the  distance  criterion(5)  does  not 
necessarily  require  computation  of  the  limiting  Q  .   For  most  applications, 
we  have  found  that  a  Q  matrix  determined  directly  via  equations  (l5),  (l6) 
and  (17),  "with  3  set  to  some  large  positive  integer,  yields  approximate 
values  of  pattern  dissimilarities  via  (5)  completely  satisfactory  for 
recognition  purposes. 

The  Transformation  Problem 

Now  assume  f  and  g  taken  from  a  set  of  patterns  normalized  with 
respect  to  intensity,  position,  and  size  but  not  standardized  with  respect  to 
orientation.   On  the  contrary,  in  this  section  we  will  assume  given  quanti- 
zations of  f  and  g,  again  (w|x)  and  (y|z)  ,  to  include  small  proper  rotational 

o 

displacements  of  individual  pattern  orientations  that  must  be  determined  and 
accounted  for  in  the  computation  of  A^   via  (5).  Therefore,  in  comparing 
f  and  g,  our  problem  is  to  determine  a  rotation  Re e  ,  the  set  of  all  proper 
rotations,  of  one  pattern  with  respect  to  the  other,  say  f  with  respect  to  g, 
yielding  an  S  via  (k)    such  that  A^   via  (5)  is  minimal  over  all  S€Z_p   as  well 
as  all  Qeit    .   This  transformation  problem  may  be  solved  in  the  following 
manner. 

Assuming  small  displacements  of  orientations  for  f  and  g,  we  may 
determine  an  initial  correspondences  matrix  Q,  via  the  method  just  presented 
using  provisionally  for  S  the  value  given  by  (U) .   Then  with  Q,  fixed,  our  new 
estimate  of  pattern  dissimilarity  may  be  formulated 

(25)  A  =  min  tr  (Q'S) 

S€Z„ 
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where  our  problem  is  again  to  determine  a  rotation  R  yielding  a  new  estimate 
of  S  via  (k)    stepwise  optimal  with  respect  to  Q. 

Define  Q,  and  _S  as  mn  X  mn  diagonals  such  that 

.(26)      (%,i'a2,2  •••'  V,mn^=  ^^1,1'%2'  ••''  \,J 
and 

(27)  (s^  ^  ,  s^  ^,  .  .  . ,  s      )=  (s     s^  „  .  .  .  ,  s    ) 
^-1,1'  —2,2'      -mn,  mn'    1^1   J- ,2  >    '  m,n 

then  we  may  express  (25)  alternatively  as 

(28)  .       A  -  min  tr  (Q'3)  =  tr  (QS) 

where  the  elements  of  S  remain  to  be  determined  as  a  function  of  the  unknown 
R. 

Also  define  W  to  be  mn  X  k  where  the  first  row  of  W  is  repeated 
n  times  as  the  first  n  rows  of  W,  the  second  row  of  W  repeated  as  the  next 
n  rows  of  W"  and  so  forth.   Define  Y  to  be  mn  X  k  where  the  entire  matrix 
Y  is  simply  repeated  vertically  m  times. 

Now  note  that  ordered  diagonal  elements  of  the  matrix  [ (WR-Y) (WR-Y)' ] 
are  identically  equal  to  the  elements  of  S^,  hence  we  may  write 

(29)  tr[(WR-Y)(WR-Y) ']  =  tr(s) 

and  since  Q_  is  also  diagonal  we  may  restate  (28)  as 

(^°^  A  =  min  trf  Q(WR-Y)  (WR-Y  ) '  ]  . 

Re0    --  -  -  - 

.1  1   jL 

Defining  ^,    such  that  Q  ^Q,^  ^  Q,  we  may  write 

1  i_ 

(31)  A.=  min  tr[^2(WR-Y)(¥R-Y)'92] 

ReG 


ind  after  manipulation, 


(32) 


,  1.         i    i         i 

A  =  min  tr  [  ig2m-^^t)'{g;^m-^Y)] 


fow  substitute  W  =  Q^^  and  Y  =  g^y  into  (32)   to  obtain 


(33) 


rhich  may  be  written 


A  =  min  tr  [  (WR-Y)'(WR-Y)] 
Re0 


(3ii)  A  =  min  tr  (R'W'WR  -  2R'W'Y  +  L'Y} 

Re0 


)r,  since  the  trace  of  a  sum  equals  the  sum  of  the  traces, 
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(35) 


A  =  min  [tr(R'W'WR)  -  2tr(R'W'Y)  +  tr(Y'Y)]. 

Ree 


Now  consider  the  first  and  last  terms  of  (35).   Clearly  both  are 
idependent  of  R.   Furthermore,  by  normalization  and  our  definitions 
E"  W,  Y,  W  and  Y  we  may  write 

1     1 

(36)  tr(R'W'WR)  =  tr(W'W)  =  tr  [  (qS^)  '  (Q^f )]  =  H  ^,-  .  I  ^?  v  = 

----    ij^'-Jk^'^ 

1    k 


nd 


(37)  tr(Y'Y)  =  tr[(Q2Y).(Q2Y)]=  I  I   q^^j  J  yf^^  =  I  ^j  I  yf^k  =  ^' 
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1    1 

Since  also  the  middle  term  of  (35)  maybe  written  -2tr  [R' (Q^l^) '  (Q  Y)]  = 
-2tr(R'W*Q,Y),  we  have  shown  that  the  transformation  problem  may  be  formulated 
equivalently  as 

(38)  A  :=  2  -  2  max  tr(R'W'QY) 

Re0 

or, 

(39)  A  =  2  -  2  max  tr(R'W'QY). 

Re0 

With  reference  to  (39)  we  notice  that  solution  of  (5)  is 
equivalent  to  solution  of 

(^0)   p    =  max  tr  (R'W'QY) 

Re0,  QeiT^ 

f.g 

where  there  exist  the  inverse  monotonic  mappings  A    =  2  -  2P    and 

P    =  1  -  7^  A^  .   Since  A    is  formulated  as  a  mean  sum  of  squared 
f,g       2  ^^S         f^g 

distances,  its  lower  bound  is  zero  hence,  the  upper  bound  for  p    is  unity. 
We  may  thus  refer  to  P„   as  the  pattern  correlation  of  f  and  g  and  solve 
(^0)  as  an  alternative  to  (5)' 

Now  if  we  allow  R  to  be  any  orthogonal  transformation,  i.e^  either 
a  proper  or  improper  rotation,  the  optimization  problem  given  by  any  of  the 

above  formulations  of  our  pattern  transformation  problem  may  be  recognized 

(  30  31 ) 
as  the  Procrustes  problem  of  psychometric s.     '     The  problem  arises  in 

factor  analysis  and  multi-dimensional  scaling  applications  where  it  is 

desirable  to  compare  two  sets  of  factor  coordinates  independently  determined 

for  the  same  set  of  variables  by  rotation  of  one  set  to  maximum  spatial 
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congruence  with  the  other  to  maximize  between-set  factor  correlations. 
Mathematically,  the  problem  is  closely  related  to  the  canonical  correlation 
problem  of  multivariate  analysis. 

It  has  been  shown  then  that  for  our  present  problem  where  W  and 
Y  and  hence  W,  Y,  W  and  Y  are  of  full-column  rank  k,  an  optimal  orthogonal 
transformation  R  is  given  by 

1 

(^1)  R  =  (HL  ^H')  W'Y 

where  H  (k  x  k)  and  L  (k  x  k  diagonal)  represent  respectively  the  eigen- 

vectors  and  the  eigenroots  of  the  matrix  (W'YY'W).  ^     ^     Since  both 

W  and  Y  are  of  rank  k,  all  roots  of  (W'YY'W)  are  positive  and  we  may  take 

as  the  elements  of  L   the  reciprocals  of  the  positive  square  roots  of 

(30) 
the  elements  of  L. 

While  it  is  true  that  computation  of  R  via  (^1)  optimizes  A  over 

all  orthogonal  transformations,  definition  of  element  correspondences  Q 

via  (U)  (or  via  (4)  where  R  has  been  chosen  as  a  proper  rotation)  makes  it 

extremely  improbable  that  the  maximum  of  tr(R'W'Q,Y)  will  now  occur  for  an 

improper  rotation--that  is,  a  reflection  of  f  about  some  axis  as  well  as  a 

proper  rotation  of  f  with  respect  to  g.   Exceptions  to  this  rule  occur  when 

comparing  patterns  whose  coordinate  matrices,  W  or  Y,  are  only  weakly  of 

full-column  rank,  i.e.  patterns  of  nominal  full-column  rank  k  whose  spatial 

geometries  can  be  accomodated  with  only  slight  distortipn  in  a  subspace  of 

dimensionality  k '  <  k.  For  example,  where  two  patterns  being  compared 

represent  quantized  left  and  right  parentheses,  "("  and  ")",  and  where  Q, 

has  been  given  by  (ll),  we  would  expect  the  R  given  by  (^1)  to  contain  a 

horizontal  reflection.  On  the  other  hand,  if  the  two  patterns  being 
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compared  are  "M"  and  "W",  both  patterns  strongly  two-dimensional,  we  would 
not  expect  an  R  computed  via  the  same  method  to  contain  a  vertical 
reflection.   In  any  case,  where  pattern  reflections  are  significant,  det  R 
may  be  computed  to  detect  improper  rotations  and  further  action  may  be 
undertaken  appropriate  to  the  specific  application. 

The  Procrustes  formulation  of  our  pattern  transformation  problem 
is  interesting,  since  it  provides  a  general  solution  applicable  for 
comparison  of  patterns  of  any  dimensionality.   In  the  case  of  planar 
pictorial  patterns,  however,  the  problem  may  be  resolved  in  a  more  direct 
fashion. 

Restricting  R  to  be  a  proper  rotation,  let  C  =  (W'QY)  for  a  given 
Q  and  write  max  tr(R'W'QY)  in  (39)  as 


fi42)   f(a)  =  max  tr 
a 


cos  a  -  sin  a 


sm  a   cos  a 


c      c   ^ 
1,1    1,2 


c      c 
2,1    2,2 


or,  equivalent ly, 

(^3)   f(a)  =  max   (c^  2  "  ^2  I'*  ^^^  ^  ^    *^^1  1  "^  ^2  2"^   ^°^  ^ 
o.  '  '  '  ' 

Then  let  A  -  (c^  2  "  ^2  l'^  ^"^^  ^  "  ^^1  1  ^  ^2  2^   ^"^^   -write  (^3)  as 

(^^)   f(a)  =  max  (a  sin  a  +  B  cos  a). 
a 

1 

2    2  2 
Also,  let  K  =  (a  +  B  )   so  that  A  =  K  sin  ^   and  B  =  K  cos  cj)  and 


(45)  f(a)  =  max  (K  sin  ^   sin  a  +  K  cos  cj)  cosa)  =  max  [K  cos  (a  -  (}>)]. 

a  ^ 

2  p  ^ 

The  maximum  occurs  when  a  =  ((>  and  is  equal  to  K  =  [(c    -  c    )  +  (c   -i+'^o  2^"^ 
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The  proper  rotation  maximizing  (39 )  is  then  determined  by  the  relations 
sin  a  =  A/K  =  (c^  ^  "  ^2  1^^^  ^^^   ^°^  a  =  B/K  =  (c^  i  +  ^2  2''/^'   ^^^^efore, 
in  comparing  any  two  planar  patterns  f  and  g,  a  proper  rotation  R,  stepwise 
optimal  with  respect  to  a  given  correspondence  matrix  Q,  can  always  be 

« 

determined  directly  as  a  function  of  the  four  elements  of  the  matrix 
C  =  (W'QY). 

In  the  last  section,  we  presented  a  method  for  determining  Q, 
optimal  with  respect  to  an  assumed  S.   In  this  section,  we  have  shown  how 
an  optimal  transformation  R,  and  hence  an  optimal  S,  may  be  determined 
with  respect  to  a  given  Q.   Since  both  subproblems  are  formulated  to 
optimize  the  same  criterion  A^   (or  P„  ),  iterative  solution  of  both 
yields  a  value  of  A_p   optimal  at  least  locally  over  all  SeZ^   and  all 
Qeir   .   Thus  given  a  set  of  quantized  patterns  G  for  which  rotational 
displacements  can  "be  assumed  small,  a  numerically  expressible  procedure 
exists  for  determining  A^   for  all  f  e  G  and  all  g  e  G. 

The  Present  Context  of  Developing  Communications  Theory 

The  particular  approach  to  pattern  communication  systems  presented 
above  results  from  a  cross-fertilization  of  two  separate  applied  mathematics 
research  efforts  conducted  by  the  author  in  the  areas  of  metropolitan 
land  use-transportation  systems  modeling  and  computer-assisted  earth 
resources  image  analysis.   Such  a  cross-fertilization  of  seemingly  divergent 
efforts  has  been  made  possible  largely  by  the  research  environment  provided 
by  the  Center  for  Advanced  Computation  (CAC)  of  the  University  of  Illinois 
at  Urbana- Champaign — a  widely  diversified  interdisciplinary  team  of 
scientists  and  technicians  seeking  appropriate  applications  of  emerging 


1 

1 

i 

1 

>4. 

> 

1 

1 

4 

\ 

/ 

/ 

8 

<4 

i 

e 

\ 

L. 

4°J° 

'i 
'^ 

^- 

';' 

J". 

C 

v". 

<^^ 

:5 

•J 

J"8 

.-J 

3 

B 

J 

H 

H 

S^ 

i 

H 

1    9 

(DOO 

8 
S 

M 

1/ 

t 

4 

*    1 
8 

H 

H 

14 

H 

5 

<•:> 

ODCP 

h 

5 

5 

s 

■3 

5 

S 

s 

C 

6 

Xi 

8"° 

6 

0 

s 

0 

S 

6- 

6"° 

°0 

% 

0 

■?; 

°7 

8 

»• 

% 

,«»>»«* 

7 

f' 

!' 

7 

"> 

? 

-I 

\ 

8 

8.^ 

% 

6' 

J'" 

0 

8% 

i'i 

P 
t* 

0 

O' 

9 

C4 

:P 

•5> 

9 

s 

.9 

^ 

6°°o 

5' 

5 

^»^5 

°   8 

c» 

0 

\} 

Q 

0 

c7 

% 

0 

0 

(1 

// 

i'"8 
8    S 

'5- 

0 

\b 

J    6 

Figure  2.   To  evaluate  the  above  numerical  procedures,  sixteen  noisy- 
copies  of  ten  prototype  patterns  (extreme  left)  were  generated.   Despite 
considerable  rotational  displacements,  all  l6o  patterns  were  correctly 
classified  using  a  Fortran  implementation  of  the  above  techniques. 


Figure  3'   A  computer  graphic  showing  the  rank  order  of  prototype-to-pattern 
dissimilarity  measures  computed  for  the  sixteen  noisy  "6's"  of  Figure  2. 
Individual  blocks  have  been  plotted  proportional  to  1/a^   •   Also,  proto- 
types have  been  ordered  from  left  to  right  in  accordance' with  overall 
prototype  similarity  with  all  example  patterns  depicted. 
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information  processing  and  conaaunication  technologies  to  problems  of 
social  concern. 

That  commionication  systems ,  especially  in  the  form  of  networks 
of  transportation  facilities,  represent  the  prime  determinants  of  regional 
and  urban  development  patterns  has  long  been  emphasized  by  geographers 
and  spatial  economists.   Since  Wiener  and  Shannon,,  moreover,  commimicatlon 
theory  concepts  have  been  embraced  enthusiastically  by  urban  analysts,  both 
for  the  construction  of  descriptive  theories  of  communities  as  evolutionary 
socioeconomic  spatial  systems  and  for  the  development  of  normative 
planning  methodologies  through  which  society  may  actively  influence  the 
determination  of  its  future  environments. 

Likening  the  various  forms  of  social  interaction  that  bind 
together  the  community  to  the  bonding  forces  that  organize  atoms  within 
molec\£Les,  molecules  within  cells,  and  cells  within  organisms,  Meier  has 
proposed  description  of  the  community  as  a  complex,  self-organizing  open 
system  whose  viability  depends  throughout  on  a  hierarchy  of  vital  processes 
struggling  to  conserve  negative  entropy.  ~^       Emphasizing  the  global  impact 
of  twentieth-century  transportation  and  communication  technologies,  and 
the  socioeconomic  specialization  that  they  afford,  Webber  has  argued  that 
our  communities  can  now  only  be  viewed  as  cultural  clusters  of  individuals 
intersecting  at  nxjmerous  levels  within  a  world  hierarchy  of  urbanization 
and  that,  within  any  comprehensive  description  of  our  communities,  non- 
spatial  structures  of  social,  economic^  and  institutional  activity  realms 

i3h) 
must  become  as  important  as  our  past  notions  of  geographic  places. 

Exploring  in  depth  the  process  of  creative  environmental  design, 

Alexander  has  proposed  a  systematic  methodology  for  planners  that  requires 
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explicit  a  priori  specification  of  all  decision  criteria  and,  where 
appropriate,  exploits  the  neutrality  of  the  computer  in  organizing  with 

respect  to  stated  values  the  planner's  progression  from  problem  analysis 

(35) 
to  solution  synthesis.      Within  the  scope  of  current  efforts  to  model 

via  computer  simulation  the  sensitivity  of  community  land  developments 

patterns  to  public  policy-making,  and  hence  to  enhance  the  rationality  of 

(36) 
public  decision-making  with  respect  to  future  urban  environments,     the 

present  author  has  pursued  mathematical  programming  techniques  that  would 

allow  a  computer-assisted  search  for  future  urban  land  use-transportation 

alternatives  maximally  congruent  with  projected  social  activity  and 

(37,38,39) 
communication  systems.  It  was  in  this  context  that  the  above 

pattern  correlation  measures  were  developed. 

To  date,  the  primary  motivation  for  pattern  information  processing 

research  at  CAC  has  been  a  cooperative  commitment  with  the  Laboratory  for 

Applications  of  Remote  Sensing  (LARS)  of  Purdue  University  to  explore 

(^0) 
applications  of  the  ILLIAC  IV  parallel  computer     at  NASA's  Ames  Research 

(Ul) 
Center  and  the  nationwide  ARPA  Network     in  support  of  the  earth  resources 

monitoring  objectives  of  the  ERTS/EROS  programs  of  NASA  and  the  U.  S. 

(I42) 
Department  of  Interior.      Multispectral  image  processing  software 

previously  researched  at  LARS  has  been  redeveloped  in  parallel  computation 

mode  to  allow  evaluation  of  the  potential  efficiencies  of  ILLIAC  IV 

interpretation  of  large  quantities  of  natural  resource,  agricultural,  and 

urban  land  use  data  now  obtainable  by  current  remote  sensing  technologies 

such  as  the  Earth  Resources  Technology  Satellite  (ERTS).   Additionally,  we 

are  exploring  the  practicalities  of  computer  communications  networking  as 

a  mechanism  to  insure  decentralized  access  by  government  resource  management 
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agencies  to  whatever  centralized  data  processing  facilities  may  prove 

efficient.   We  hope  to  explore  the  applicability  of  the  pattern  correlation 

techniques  presented  above  to  a  number  of  data  processing  tasks  within 

this  research. 

Throughout  all  of  this  work,  we  find  ourselves  living  Wiener's 

thesis  of  I95O: 

". . .  that  society  can  only  be  imderstood  through 
a  study  of  the  messages  and  the  communication 
facilities  which  belong  to  it;  and  that  in  the 
future  development  of  these  messages  and  communi- 
cation facilities,  messages  between  man  and 
machines,  between  machines  and  man,  and  between 
machine  and  machine,  are  destined  to  play  an 
ever-increasing  part. "  (  3  j 

The  mechanical  Information  processing  and  communication 
technologies  in  service  to  man  today  rest  firmly  on  the  foundations  of 
encoded  Information  theory  established  by  Wiener  and  Shannon  more  than 
two   and  a  half  decades  ago.  Much  of  our  current  pattern  information 
processing  research  is  motivated  by  a  continued  enthusiasm  to  comprehend 
better  the  nature  of  human  communication  processes  in  order  to  be  able 
to  program  the  machine  to  simulate — in  some  primitive  fashion — under- 
standing of  man's  own  communications  and  thus  to  make  convenient  the 
machine's  subservience  to  man  on  man's  own  terms.   A  more  important  outcome 
of  this  research,  we  think,  will  be  a  more  complete  appreciation  for 
the  unique  characteristics  and  capacities  of  human  communications  and  an 
acute   recognition  of  the  primacy  of  human  communications  for  the 
continued  realization  of  the  potentialities  of  man. 

From  the  standpoint  of  modern  psychology,  the  present  model  of 
pattern  commimications  seems  promising  in  that  it  suggests  possible  new 
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approaches    for  development   of  macro  mathematical  theories   of  the  information 
encoding,    transmission,    and  decoding  processes  that  take  place  within  our 
own  brains   and  hence   control  the   complex  organization  of  pattern   information 
we  experience   as   language,    thought,    and  awareness. 

Since  our  model   defines  patterns   as   distributions   of  quantized 
information  over   sets   of  discrete,    information  storage  elements,  by- 
analogy,   we  may   speculate  the  pattern   information  of  the  brain  to  exist 
as  pattern-specific  biochemical   substances   distributed  throughout   intricate 
networks   of  neurons.      Such  a  conceptualization  of  information  within  the 
brain  would  seem  apt   not  only   for  representation  of  afferent   and  efferent 
memory  traces  within  the  modal   centers   of  the  brain,   but    also   for  repre- 
sentation of  complex  internuncial   associations   of  patterns  within  and 
between  modal   areas  throughout  highly  structured  pathways   as  well  as 
random  cortical  networks. 

The  possibility  that  the  processes   of  information  organization 
within  the  brain  that   give   rise  to  language   and  thought,    however  parallel 
and  complex,   might   be   scientifically  demonstrated  to  be   governed  by  some 
dynamic   compromise  between  the  natural  laws   of  energy  conservation  and 
entropy  maximization — concepts    so    fundamental  to   our  present  understanding 
of  the  physical  world  around  us   and  becoming  increasingly  important   for 
our   description  of  our   social  environment — is   indeed  exciting.      Such   a 
demonstration   should  contribute   greatly  toward  man's   confidence   in  the 
uniqueness  within  nature   of  his    own   capacity   for  information   acquisition, 
assimilation,    and   organization--pattern   synthesis--a  capacity  then 
mathematically  expressible  only  as   some   joint  product  of  the   factorial 
complexity  of  possible   synaptic   transmissions   and   the  enormous   capacities 
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for  information  coding  particular  biochemical  substances  within  the  brain 

(i^3,UU,i+5) 
are  known  to  process. 

Referencing  continually  throughout  his  own  work  the  ideas  of  men 

such  as  Leibniz,  Lock^  and  Bergson,  Wiener  viewed  our  present  paradigm  of 

cybernetics  simply  as  a  logical  continuation  of  man's  universal  inquiry 

into  the  nature  of  his  own  being  and  his  destiny.   Suimnarizing  a  succession 

of  individual  perspectives  on  human  commimication,  Matson  and  Montagu 

have  more  recently  termed  this  inquiry  the  "unfinished  revolution"  of 

communications.      That  present  communication  technologies  now  make 

practical  for  this  inquiry  a  platform  of  planetairy  dimensions,  and  that 

our  theories  of  community  and  communications  continue  to  develop  in 

harmony,  perhaps  both  as  extensions  of  a  natural  psychology  of  man,  give 

us  much  hope  that  this  inquiry  will  be  carried  into  the  future  by  and 

for  a  global  order  of  man,  an  order  continually  rising  above  a  diversity 

of  cultural  languages  and  national  institutions.  As  we  seek  for  our  own 

present  generation  participation  within  this  social  dialogue,  we  must 

proceed  with  a  reverence  for  human  commimication  similar  to  that  expressed 

by  Dewey: 

"Communication  is  uniquely  instrumental  and 
uniquely  final.   It  is  Instrumental  as 
liberating  us  from  the  otherwise  overwhelming 
pressure  of  events  and  enabling  us  to  live  in 
a  world  of  things  that  have  meaning.   It  is 
final  as  a  sharing  in  the  objects  and  arts 
precious  to  a  community,  a  sharing  whereby 
meanings  are  enhanced,  deepened  and  solidified 
in  the  sense  of  communion. . .When  the  instru- 
mental and  final  functions  of  communication 
live  together  in  experience,  there  exists  an 
intelligence  which  is  the  method  and  reward 
of  the  common  life,  and  a-  society  worthy  to 
command  affection,  admiration,  and  loyalty,  "^^i-' 


27 
Acknowledgements 

The  author  wishes  to  express  his  appreciation  for  the 
Interdisciplinary  support  provided  by  numerous  colleagues  at  the  Center 
for  Advanced  Computation  in  the  preparation  of  this  paper.   Special 
acknowledgements  are  due  Professor  Daniel  L.  Slotnick  for  numerous 
discussions  concerning  communication  systems  in  general,  as  well  as  for 
introducing  the  author  to  the  potential  advantages  of  parallel  processing 
concepts  for  social  systems  modeling.   The  Algorithm  Development  Group  of 
CAC  offered  much  assistance  with  mathematical  formulations  and  notation. 
For  example,  the  special  solution  to  the  planar  orthogonal  rotation 
problem  was  pointed  out  to  the  author  "by  Dr.  George  B.  Purdy. 


28 

References 

1.  Rosenblueth,  A.,  Wiener,  N.,  and  Bigelow,  J.,  "Behavior,  Purpose,  and 

Teleology,  "  Philosophy  of  Science,  Vol.  10,  19^3,  pp.  l8-2l^■. 

2.  Wiener,  N.,  Cybernetics,  The  Technology  Press  of  M.I.T.  and  John 

Wiley  and  Sons,  Inc.,  New  York,  19^1-8. 

3.  Wiener,  N.,  The  Human  Use  of  Human  Beings,  Houghton  Mifflin,  Boston, 

1950. 
h.      Shannon,  C  E.,  "A  Mathematical  Theory  of  Communication,  "  Bell  System 
Technical  Journal,  Vol.  27,  19^8,  pp.  379-^23,  623-656. 

5.  Weaver,  W.,  "Recent  Contributions  to  the  Mathematical  Theory  of 

Communication,  "  in  Shannon,  C  E.,  and  Weaver,  W.,  The 
Mathematical  Theory  of  Communication,  University  of  Illinois 
Press,  Urbana,  19^9* 

6.  Quastler,  H.  (ed.).  Information  Theory  in  Psychology  (Proceedings  of 

a  Conference  on  the  Estimation  or  Information  Flow,  Monticello, 
Illinois,  July  5-9^  195^)^  The  Free  Press,  Glencoe,  Illinois, 

1955. 
7'   Garner,  W.  R.,  Uncertainty  and  Structure  as  Psychological  Concepts, 
Wiley,  New  York,  I962. 

8.  Miller,  G.  A.,  Galanter,  E.,  and  Pribram,  K.  H.,  Plans  and  the 

Structure  of  Behavior,  Holt,  Rinehart  and  Winston,  Inc.,  New  York, 
i960. 

9.  Miller,  G.  A.,  The  Psychology  of  Communication,  Basic  Bopks,  Inc., 

1967. 
10.   Chomsky,  N.,  Syntactic  Structures,  Mduton  and  Co.,  The  Hague,  1957. 


29 

11.  MorriS;  C,  Signification  and  Significance,  The  M.I.T.  Press, 

Cambridge,  196^. 

12.  Cherry,  C,  On  Human  Communication:   A  Review,  A  Survey,  and 

A  Criticism  (3rd  ed.).  The  M.I.T.  Press,  Cambridge,  196ij-. 

13.  Pierce,  J.  R.,  "Communication,  "  Scientific  American,  Vol.  227, 

No.  3,  September  1972,  pp.  31-iil.  - 

14.  Watanabe,  S.  (ed.).  Methodologies  of  Pattern  Recognition,  Academic 

Press,  New  York,  1969. 

15.  Watanabe,  S.  (ed.).  Frontiers  of  Pattern  Recognition,  Academic 

Press,  New  York,  1972. 

16.  Widrow,  B.,  "The  Rubber-Mask  Technique  --  Pattern  Measurement  and 

Analysis,  "  Journal  of  the  Pattern  Recognition  Society,  Vol.  5; 
No.  3,  September  I973,  pp.  I75-I97. 

17.  Widrow,  B.,  "The  Rubber-Mask  Technique  --  Pattern  Storage  and 

Recognition,  "  Journal  of  the  Pattern  Recognition  Society, 
Vol.  5,  No.  3,  September  I973,  pp.  199-211. 

18.  Selfrldge,  0.  G.,  and  Neisser,  U.,  "Pattern  Recognition  by 

Machine,  "  Scientific  American,  Vol.  203,  No.  I8,  August  I960, 
pp.  60-79i  reprinted  in  Feigenbaum,  E.,  and  Feldman,  J.  (eds.). 
Computers  and  Thought,  McGraw-Hill  Book  Co.,  New  York,  1963* 
19-  Minsky,  M.,  "Steps  Toward  Artificial  Intelligence,  "  Proceedings  of 
the  Institute  of  Radio  Engineers,  Vol.  k9,    January  I96I, 
pp.  8-3O;  reprinted  in  Feigenbaum,  E.,  and  Feldman,  J.  (eds.). 
Computers  and  Thought,  McGraw-Hill  Book  Co.,  New  York,  1963* 


30 


20.  Nagy,  G.,  "State  of  the  Art  in  Pattern  Recognition,  "  Proceedings  of 

the  IEEE,  Vol.  56,  No.  5,  May  I968,  pp.  836-862. 

21.  Hitchcock,  F.  L.,  "The  Distribution  of  a  Product  from  Several 

Sources  to  Numerous  Localities,  "  Journal  of  Mathematics  and 
Physics,  Vol.  20,  19^1,  pp.  22J^-230. 

22.  Dorfman,  R.,  Samuelson,  P.  A.,  and  Solow,  R.  M.,  Linear  Programming 

and  Economic  Analysis,  McGraw-Hill  Book  Co.,  New  York,  1958* 

23.  Dantzig,  G.  B.,  Linear  Programming  and  Extensions,  Princeton 

University  Press,  Princeton,  1963* 
2U.   Wilson,  A.  G.,  Entropy  in  Urban  and  Regional  Modelling,  Pion 
Limited,  London,  1970* 

25.  Potts,  R.  B.,  and  Oliver,  R.  M.,  Flows  in  Transportation  Networks, 

Academic  Press,  New  York,  1972. 

26.  Evans,  A.  W.,  "Some  Properties  of  Trip  Distribution  Methods,  " 

Transportation  Research,  Vol.  k,    1970^  PP«  19-36. 

27.  Evans,  A.  W.,  "The  Calibration  of  Trip  Distribution  Models  with 

Exponential  or  Similar  Cost  Functions,  "  Transportation  Research, 
Vol.  5,  1971,  pp.  15-38. 

28.  Evans,  S.  P.,  "A  Relationship  between  the  Gravity  Model  for  Trip 

Distribution  and  the  Transportation  Problem  in  Linear 
Programming,  "  Transportation  Research,  Vol.  7,  1973,  PP*  39-61. 

29.  Bacharach,  M.,  Biproportional  Matrices  and  Input-Output  Change, 

Cambridge  University  Press,  Cambridge,  1970. 

30.  Schonemann,  P.  H.,  "A  Generalized  Solution  of  the  Orthogonal 

Procrustes  Problem,  "  Psychometrika,  Vol.  3I;  No.  1,  March  I966, 
pp.  1-10. 


31 

31.  Cliff,  N.,  "Orthogonal  Rotation  to  Congruence,  "  Psychometrika,  Vol.  3I, 

No.  1,  March  I966,  pp.  33-^2. 

32.  Green,  B.  F.,  "The  Orthogonal  Approximation  of  an  Oblique  Structure 

in  Factor  Analysis,  "  Psychometrika,  Vol.  I7,  No.  k,    December  1952, 
pp.  i^29-i^40. 

33.  Meier,  R.  L.,  A  Communications  Theory  of  Urban  Growth,  The  M.I.T. 

Press,  Cambridge,  I962. 

3^1-.   Webber,  M.  M.,  "The  Urban  Place  and  the  Nonplace  Urban  Realm,  "in 
Webber,  M.  M.,  et  al..  Explorations  into  Urban  Structure, 
University  of  Pennsylvania  Press,  Philadelphia,  196^>  PP«  79-153* 

35-   Alexander,  C,  Notes  on  the  Synthesis  of  Form,  Harvard  University 
Press,  Cambridge,  I966. 

36.   Heramens,  G.  C.  (ed.).  Urban  Development  Models,  Special  Report  91} 

Highway  Research  Board,  National  Academy  of  Sciences,  2101  Consti- 
tution Avenue,  Washington,  D.  C,  I968. 

37'   Ray,  R.  M.,  "Heuristic  Spatial  Allocation,  "  Urban  System  Latoratory 
Working  Paper,  Department  of  City  and  Regional.  Planning,  Univer- 
sity of  North  Carolina  At  Chapel  Hill,  Chapel  Hill,  North 
Carolina  2751^,  January  I969.   (mimeo) 

38.   Ray,  R.  M.,  "A  Conceptual  Framework  for  Urban  Spatial  Planning,  " 
Urban  System  Laboratory  Working  Paper,  Department  of  City  and 
Regional  Planning,  University  of  North  Carolina  at  Chapel  Hill, 
North  Carolina  2751^+,  May  1970.   (mimeo) 

39-  Ray,  R.  M.,  "A  Generalized  ILLIAC  IV  Urban  Land  Use  Planning  Model,  " 
CAC  Document  No.  5I,  Center  for  Advanced  Computation,  University 
of  Illinois  at  Urbana- Champaign,  Urbana,  Illinois  6180I,  October 
1972. 


32 

40.   Slotnick,  D.  L.,  "The  Fastest  Computer,  "  Scientific  American^  Vol.  224, 
No.  2,  February  1971^  PP-  76-87. 

ill.   Roberts,  L.  G.,  and  Wessler,  B.  D.,  "Computer  Network  Development  to 
Achieve  Resource  Sharing,  "  AFIPS  Proceedings,  1970  Spring  Joint 
Computer  Conference,  Vol.  36,  AFIPS  Press,  1970,  pp.  5^3- 5^9 • 

42.   Ray,  R.  M.,  Thomas,  J.  D.,  Donovan,  W.  E.,  and  Swain,  P.  H., 

"Implementation  of  ILLIAC  IV  Algorithms  for  Multispectral  Image 
Interpretation,  Final  Report,  Covering  the  Period  March  1973  - 
February  197^,  "  CAC  Document  No.  112,  Center  for  Advanced 
Computation,  University  of  Illinois  at  Urbana-Champaign,  Urbana, 
Illinois  61801,  June  197^. 

k3'      Koffka,  K.,  Principles  of  Gestalt  Psychology,  Harcourt,  Brace  and 
World,  Inc.,  New  York,  1935- 

44.  Deese,  J.,  The  Structure  of  Associations  in  Language  and  Thought, 

John  Hopkins  Press,  Baltimore,  1965* 

45.  Rose,  S.,  The  Conscious  Brain,  Alfred  A.  Knopf,  Inc.,  New  York,  1973. 

46.  Matson,  F.  W.,  and  Montagu,  A.,  "Introduction:   The  Unfinished 

Revolution,  "  in  Matson,  F.  W.,  and  Montagu,  A.,  The  Human 
Dialogue  --Perspectives  on  Communication,  The  Free  Press, 
New  York,  I967. 
47-   Dewey,  J.,  Experience  and  Nature  (2nd  ed.),  The  Open  Court  Publishing 
Co.,  La  Salle,  Illinois,  I929. 


